Experiences in Tuning Performance of Hybrid MPI/OpenMP Applications on Quad-core Systems

The Hybrid method of parallelization (using MPI for internode communication and OpenMP for intra-node communication) seems a natural fit for the way most clusters are built today. It is generally expected to help programs run faster due to factors like availability of greater bandwidth for intra-node communication. However, optimizing hybrid applications for maximum speedup is difficult primarily due to inadequate transparency provided by the OpenMP constructs and also due to the dependence of the resulting speedup on the combination in which MPI and OpenMP is used. In this paper we mention some of our experiences in trying to optimize applications built using MPI and OpenMP. More specifically, we talk about the different techniques that could be helpful to other researchers working on hybrid applications. To demonstrate the usefulness of these optimizations, we provide results from optimizing a few typical scientific applications. Using these optimizations, one hybrid code ran up to 34% faster than pure-MPI code.